Investigation of the topography-dependent current in conductive AFM and the calibration method

The topography and the electrical properties of materials are two crucial characteristics in determining their functionalities. Conductive atomic force microscopy (CAFM) is widely recognized for its ability to independently measure the topology and conductivity of the sample surface. The increasing trend towards miniaturization in electrical devices and sensors has led to an urgent demand for enhancing the accuracy of CAFM characterization. However, the sample's topography may affect the current measured by CAFM, leading to an inaccurate estimation of the sample's conductivity. Herein, we investigated the existence of topography-dependent current that originates from changes in capacitance between the probe and sample in CAFM testing. A linear correlation between the current and topography has been established using both experimental and theoretical methods. A calibration method based on this linear correlation has been proposed to eliminate the current error induced by the uneven surface of both insulators and conductors. This work will yield substantial advantages for research requiring high-precision CAFM testing.Comment: Corrected typo

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

The great success of Transformer-based models benefits from the powerful multi-head self-attention mechanism, which learns token dependencies and encodes contextual information from the input. Prior work strives to attribute model decisions to individual input features with different saliency measures, but they fail to explain how these input features interact with each other to reach predictions. In this paper, we propose a self-attention attribution method to interpret the information interactions inside Transformer. We take BERT as an example to conduct extensive studies. Firstly, we apply self-attention attribution to identify the important attention heads, while others can be pruned with marginal performance degradation. Furthermore, we extract the most salient dependencies in each layer to construct an attribution tree, which reveals the hierarchical interactions inside Transformer. Finally, we show that the attribution results can be used as adversarial patterns to implement non-targeted attacks towards BERT.Comment: AAAI-202

Feature Extraction and Classification of EHG between Pregnancy and Labour Group Using Hilbert-Huang Transform and Extreme Learning Machine

Preterm birth (PTB) is the leading cause of perinatal mortality and long-term morbidity, which results in significant health and economic problems. The early detection of PTB has great significance for its prevention. The electrohysterogram (EHG) related to uterine contraction is a noninvasive, real-time, and automatic novel technology which can be used to detect, diagnose, or predict PTB. This paper presents a method for feature extraction and classification of EHG between pregnancy and labour group, based on Hilbert-Huang transform (HHT) and extreme learning machine (ELM). For each sample, each channel was decomposed into a set of intrinsic mode functions (IMFs) using empirical mode decomposition (EMD). Then, the Hilbert transform was applied to IMF to obtain analytic function. The maximum amplitude of analytic function was extracted as feature. The identification model was constructed based on ELM. Experimental results reveal that the best classification performance of the proposed method can reach an accuracy of 88.00%, a sensitivity of 91.30%, and a specificity of 85.19%. The area under receiver operating characteristic (ROC) curve is 0.88. Finally, experimental results indicate that the method developed in this work could be effective in the classification of EHG between pregnancy and labour group

Moderate-to-vigorous intensity physical activity levels of children with intellectual disability during physical education classes

BackgroundPhysical education (PE) class is an excellent way to improve moderate-to-vigorous intensity physical activity (MVPA). Increasing number of research has explored the children’s PA based on movement during PE classes, but data for children with intellectual disability (ID) is still lacking.PurposeThe purpose of this study was to investigate the current status of MVPA levels of children with ID during PE classes in China, as well as differences of MVPA levels according to gender and grade.MethodsAccelerometers were used to record MVPA levels of fifty-three children with severe ID from 9 to 16 years of age (mean age: 12.60 ± 1.66 years) during standard PE classes.ResultsThe mean time spent in MVPA during PE classes was 8.00 ± 2.10 min, meaning only 22.88% of PE class time was spent in MVPA. As grade levels progresses, time spent in MVPA during PE classes tended to decrease; the fourth-grade children tended to spend more time in MVPA during PE classes compared with the fifth-grade and the sixth-grade (9.15 vs. 7.61 vs. 7.25 min, all p < 0.05). Boys spend significantly more time in MVPA during PE classes than girls; both in the entire sample (9.20 vs. 5.70 min) as well as in each grade (9.76 vs. 6.09 min, 9.35 vs. 5.68 min, 8.31 vs. 5.59 min, all p < 0.05).ConclusionFindings from this study indicate that the proportion of PE class spent in the MVPA of children with ID was lower than the 50% recommended by the U.S. Department of Health and Human Services (DHHS) and U.K. Association for Physical Education (AfPE). And the amount of MVPA participation varied by the grade and gender as well as by the activity performed. Therefore, in order to help children with ID achieve MVPA goals, educators need to reevaluate the PE curriculum as well as take due consideration of grade and gender when devising new content

Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes

In this paper, we move towards combining large parametric models with non-parametric prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework for fine-tuning pretrained language models (LM), which automatically learns a bias to improve predictive performance for varying data sizes, especially low-resource settings. Our prototypical fine-tuning approach can automatically adjust the model capacity according to the number of data points and the model's inherent attributes. Moreover, we propose four principles for effective prototype fine-tuning towards the optimal solution. Experimental results across various datasets show that our work achieves significant performance improvements under various low-resource settings, as well as comparable and usually better performances in high-resource scenarios.Comment: Published as a conference paper at AAAI 202

Kosmos-2: Grounding Multimodal Large Language Models to the World

We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.g., bounding boxes) and grounding text to the visual world. Specifically, we represent refer expressions as links in Markdown, i.e., ``[text span](bounding boxes)'', where object descriptions are sequences of location tokens. Together with multimodal corpora, we construct large-scale data of grounded image-text pairs (called GrIT) to train the model. In addition to the existing capabilities of MLLMs (e.g., perceiving general modalities, following instructions, and performing in-context learning), Kosmos-2 integrates the grounding capability into downstream applications. We evaluate Kosmos-2 on a wide range of tasks, including (i) multimodal grounding, such as referring expression comprehension, and phrase grounding, (ii) multimodal referring, such as referring expression generation, (iii) perception-language tasks, and (iv) language understanding and generation. This work lays out the foundation for the development of Embodiment AI and sheds light on the big convergence of language, multimodal perception, action, and world modeling, which is a key step toward artificial general intelligence. Data, demo, and pretrained models are available at https://aka.ms/kosmos-2.Comment: 20 page

RepBNN: towards a precise Binary Neural Network with Enhanced Feature Map via Repeating

Binary neural network (BNN) is an extreme quantization version of convolutional neural networks (CNNs) with all features and weights mapped to just 1-bit. Although BNN saves a lot of memory and computation demand to make CNN applicable on edge or mobile devices, BNN suffers the drop of network performance due to the reduced representation capability after binarization. In this paper, we propose a new replaceable and easy-to-use convolution module RepConv, which enhances feature maps through replicating input or output along channel dimension by $\beta$ times without extra cost on the number of parameters and convolutional computation. We also define a set of RepTran rules to use RepConv throughout BNN modules like binary convolution, fully connected layer and batch normalization. Experiments demonstrate that after the RepTran transformation, a set of highly cited BNNs have achieved universally better performance than the original BNN versions. For example, the Top-1 accuracy of Rep-ReCU-ResNet-20, i.e., a RepBconv enhanced ReCU-ResNet-20, reaches 88.97% on CIFAR-10, which is 1.47% higher than that of the original network. And Rep-AdamBNN-ReActNet-A achieves 71.342% Top-1 accuracy on ImageNet, a fresh state-of-the-art result of BNNs. Code and models are available at:https://github.com/imfinethanks/Rep_AdamBNN.Comment: This paper has absolutely nothing to do with repvgg, rep means repeatin

Case report: Sarcomatoid urothelial carcinoma of the renal pelvis masquerading as a renal abscess

Sarcomatoid urothelial carcinoma (SUC), a rare tumor of the urinary tract epithelium, exhibits a high degree of malignancy and therefore a poor prognosis. Due to the absence of specific clinical presentations and imaging findings, SUC of the renal pelvis masquerades as a renal abscess is frequently under-recognized or misdiagnosed as benign inflammatory disease, resulting in delayed or erroneous treatment. Here, we report a patient with SUC of the renal pelvis who presented with a renal abscess. Repeated anti-inflammatory treatment was ineffective. Unexpectedly, cancerous cells were detected in subsequent exfoliative cytology of nephrostomy drainage fluid. In accordance with this, radical surgery and postoperative chemotherapy were conducted. Fortunately, neither recurrence nor metastasis occurred during a one-year follow-up